Fix locale-dependent JSON float parsing (Issue #241) #255
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
strtod() respects LC_NUMERIC locale, causing JSON parsing to fail in non-C locales (French, German, etc.) where comma is the decimal separator. Implemented custom locale-independent strtod_c() parser: - Always uses '.' as decimal separator per JSON spec - Handles sign, integer, fractional, and exponent parts - No platform dependencies or thread-safety issues - Simple and portable (~87 lines) Added test_vec0_locale_independent() to verify parsing works under non-C locales. All tests pass (73 passed, 4 skipped). Fixes asg017#241 and asg017#168
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR syncs a commit ported from an upstream fork into this repo to fix locale-dependent JSON float parsing.
Original commit
strtod()usage when parsing JSON numeric tokens with a locale-independentstrtod_c()implementation, ensuring JSON floats are always parsed using.as the decimal separator (per JSON spec).Why it fits this repo’s guidelines/policies
.as the decimal separator; parsing should not vary based onLC_NUMERIC.strtod_l()/ global locale switching, which can be inconsistent across platforms and tricky for thread-safety.Notes (if any adaptations):
Why it’s useful here
Clocales (e.g. French/German).Testing
uv sync --directory tests(pass)make test-loadable python=./tests/.venv/bin/python(pass)Cnumeric locale and verifies JSON float parsing remains correct.